83 research outputs found
Deep Learning Guided Autonomous Surgery: Guiding Small Needles into Sub-Millimeter Scale Blood Vessels
We propose a general strategy for autonomous guidance and insertion of a
needle into a retinal blood vessel. The main challenges underpinning this task
are the accurate placement of the needle-tip on the target vein and a careful
needle insertion maneuver to avoid double-puncturing the vein, while dealing
with challenging kinematic constraints and depth-estimation uncertainty.
Following how surgeons perform this task purely based on visual feedback, we
develop a system which relies solely on \emph{monocular} visual cues by
combining data-driven kinematic and contact estimation, visual-servoing, and
model-based optimal control. By relying on both known kinematic models, as well
as deep-learning based perception modules, the system can localize the surgical
needle tip and detect needle-tissue interactions and venipuncture events. The
outputs from these perception modules are then combined with a motion planning
framework that uses visual-servoing and optimal control to cannulate the target
vein, while respecting kinematic constraints that consider the safety of the
procedure. We demonstrate that we can reliably and consistently perform needle
insertion in the domain of retinal surgery, specifically in performing retinal
vein cannulation. Using cadaveric pig eyes, we demonstrate that our system can
navigate to target veins within 22 XY accuracy and perform the entire
procedure in less than 35 seconds on average, and all 24 trials performed on 4
pig eyes were successful. Preliminary comparison study against a human operator
show that our system is consistently more accurate and safer, especially during
safety-critical needle-tissue interactions. To the best of the authors'
knowledge, this work accomplishes a first demonstration of autonomous retinal
vein cannulation at a clinically-relevant setting using animal tissues
Optical Fiber-Based Needle Shape Sensing in Real Tissue: Single Core vs. Multicore Approaches
Flexible needle insertion procedures are common for minimally-invasive
surgeries for diagnosing and treating prostate cancer. Bevel-tip needles
provide physicians the capability to steer the needle during long insertions to
avoid vital anatomical structures in the patient and reduce post-operative
patient discomfort. To provide needle placement feedback to the physician,
sensors are embedded into needles for determining the real-time 3D shape of the
needle during operation without needing to visualize the needle
intra-operatively. Through expansive research in fiber optics, a plethora of
bio-compatible, MRI-compatible, optical shape-sensors have been developed to
provide real-time shape feedback, such as single-core and multicore fiber Bragg
gratings. In this paper, we directly compare single-core fiber-based and
multicore fiber-based needle shape-sensing through identically constructed,
four-active area sensorized bevel-tip needles inserted into phantom and \exvivo
tissue on the same experimental platform. In this work, we found that for
shape-sensing in phantom tissue, the two needles performed identically with a
-value of , but in \exvivo real tissue, the single-core fiber
sensorized needle significantly outperformed the multicore fiber configuration
with a -value of . This paper also presents the experimental
platform and method for directly comparing these optical shape sensors for the
needle shape-sensing task, as well as provides direction, insight and required
considerations for future work in constructively optimizing sensorized needles
Distributed fiber optics 3D shape sensing by means of high scattering NP-doped fibers simultaneous spatial multiplexing
International audienceA novel approach for fiber optics 3D shape sensing, applicable to mini-invasive bio-medical devices, is presented. The approach exploits the optical backscatter reflectometry (OBR) and an innovative setup that permits the simultaneous spatial multiplexing of an optical fibers parallel. The result is achieved by means of a custom-made enhanced backscattering fiber whose core is doped with MgO-based nanoparticles (NP). This special NP-doped fiber presents a backscattering-level more than 40 dB higher with respect to a standard SMF-28. The fibers parallel is built to avoid overlap between NP-doped fibers belonging to different branches of the parallel, so that the OBR can distinguish the more intense backscattered signal coming from the NP-doped fiber. The system is tested by fixing, with epoxy glue, 4 NP-doped fibers along the length of an epidural needle. Each couple of opposite fibers senses the strain on a perpendicular direction. The needle is inserted in a custom-made phantom that simulates the spine anatomy. The 3D shape sensing is obtained by converting the measured strain in bending and shape deformation
Deep Learning Guided Autonomous Retinal Surgery using a Robotic Arm, Microscopy, and iOCT Imaging
Recent technological advancements in retinal surgery has led to the modern
operating room consisting of a surgical robot, microscope, and intraoperative
optical coherence tomography (iOCT). The integration of these tools raises the
fundamental question of how to effectively combine them to enable surgical
autonomy. In this work, we address this question by developing a unified
framework that enables real-time autonomous surgical workflows utilizing the
aforementioned devices. To achieve this, we make the following contributions:
(1) we develop a novel imaging system that integrates microscopy and iOCT in
real-time, accomplished by dynamically tracking the surgical instrument via a
small iOCT scanning region (e.g. B-scan), which was not previously possible;
(2) implementing various convolutional neural networks (CNN) that automatically
segment and detect task-relevant information for surgical autonomy; (3)
enabling surgeons to intuitively select goal waypoints within both the
microscope and iOCT views through simple mouse-click interactions; (4)
integrating model predictive control (MPC) for real-time trajectory generation
that respects kinematic constraints to ensure patient safety. We show the
utility of our system by tackling subretinal injection (SI), a challenging
procedure that involves inserting a microneedle below the retinal tissue for
targeted drug delivery, a task surgeons find challenging due to requiring
tens-of-micrometers of accuracy and precise depth perception. We validate our
system by conducting 30 successful SI trials on pig eyes, achieving needle
insertion accuracy of to various subretinal goals and
duration of seconds. Preliminary comparisons to a human operator
performing SI in robot-assisted mode highlight the enhanced safety of our
system.Comment: pending submission to a journa
Quantitative Bioluminescence Tomography-guided System for Conformal Irradiation In Vivo
Although cone-beam CT (CBCT) has been used to guide irradiation for
pre-clinical radiotherapy(RT) research, it is limited to localize soft tissue
target especially in a low imaging contrast environment. Knowledge of target
shape is a fundamental need for RT. Without such information to guide
radiation, normal tissue can be irradiated unnecessarily, leading to
experimental uncertainties. Recognition of this need led us to develop
quantitative bioluminescence tomography (QBLT), which provides strong imaging
contrast to localize optical targets. We demonstrated its capability of guiding
conformal RT using an orthotopic bioluminescent glioblastoma (GBM) model. With
multi-projection and multi-spectral bioluminescence imaging and a novel
spectral derivative method, our QBLT system is able to reconstruct GBM with
localization accuracy <1mm. An optimal threshold was determined to delineate
QBLT reconstructed gross target volume (GTV_{QBLT}), which provides the best
overlap between the GTV_{QBLT} and CBCT contrast labeled GBM (GTV), used as the
ground truth for the GBM volume. To account for the uncertainty of QBLT in
target localization and volume delineation, we also innovated a margin design;
a 0.5mm margin was determined and added to GTV_{QBLT} to form a planning target
volume (PTV_{QBLT}), which largely improved tumor coverage from 75% (0mm
margin) to 98% and the corresponding variation (n=10) of the tumor coverage was
significantly reduced. Moreover, with prescribed dose 5Gy covering 95% of
PTV_{QBLT}, QBLT-guided 7-field conformal RT can irradiate 99.4 \pm 1.0% of GTV
vs. 65.5 \pm 18.5% with conventional single field irradiation (n=10). Our
QBLT-guided system provides a unique opportunity for researchers to guide
irradiation for soft tissue targets and increase rigorous and reproducibility
of scientific discovery
- …